18.S096: Compressed Sensing and Sparse Recovery

Most of us have noticed how saving an image in JPEG dramatically reduces the space it occupies in our hard drives (as oppose to file types that save the pixel value of each pixel in the image). The idea behind these compression methods is to exploit known structure in the images; although our cameras will record the pixel value (even three values in RGB) for each pixel, it is clear that most collections of pixel values will not correspond to pictures that we would expect to see. This special structure tends to exploited via sparsity. Indeed, natural images are known to be sparse in certain bases (such as the wavelet bases) and this is the core idea behind JPEG (actually, JPEG2000; JPEG uses a different basis). Let us think of x ∈ RN as the signal corresponding to the image already in the basis for which it is sparse. Let’s say that x is s-sparse, or ‖x‖0 ≤ s, meaning that x has, at most, s non-zero components and, usually, s N . The `0 norm1 ‖x‖0 of a vector x is the number of non-zero entries of x. This means that, when we take a picture, our camera makesN measurements (each corresponding to a pixel) but then, after an appropriate change of basis, it keeps only s N non-zero coefficients and drops the others. This motivates the question: “If only a few degrees of freedom are kept after compression, why not measure in a more efficient way and take considerably less than N measurements?”. This question is in the heart of Compressed Sensing [CRT06a, CRT06b, CT05, CT06, Don06, FR13]. It is particularly important in MRI imaging [?] as less measurements potentially means less measurement time. The following book is a great reference on Compressed Sensing [FR13]. More precisely, given a s-sparse vector x, we take s < M N linear measurements yi = ai x and the goal is to recover x from the underdetermined system: